FIELD OF THE INVENTION
The present invention relates generally to a neuro-computer having a plurality of neurons (processors) and a method of using the same, and particularly to a method of generalizing and speeding up a neuro-computer. The invention is applicable to a neuro-computer and an information processing system using the same, for example, for the knowledge processing fields such as recognition, prediction, inspection, analysis, and control.
Two types of conventional neuro-computers are explained in the following.
A first conventional technique will be first given. This first conventional technique has been proposed by the present applicant in which presented is a complete digital neuron using WSI (Wafer Scale Integration). This technique is described in "Technical Study Report", pp. 55-60, the Integrated Circuit Study Group (ICD 89-151, 1989, November) of the Institute of Electronics, Information and Communications Engineers. An example of a neuron of the complete digital neuro-computer is shown in FIG. 8. Referring to FIG. 8, a multiplier 3810 corresponds to a synapse which multiplies a data input signal supplied from another neuron by a synapse weight stored in a memory 3840. The address of a synapse weight in the memory 3840 is given by an address signal. The multiplied result is applied to a loop circuit made of an adder/subtracter 3820 and a latch 3860, and is cumulatively added each time a step signal is inputted. If an underflow and the like occurs, a shifter 3830 is used when necessary. When addresses designated by address signals circulate once, in response to a cycle signal the contents of the latch 3860, i.e., the cumulative addition result, is transferred to a latch 3870 and used as an output of this neuron at the next cycle. When an address signal designates this neuron, a driver 3880 is enabled by a decoder 3890 to deliver a data output signal. A sigmoid converter and the like are provided on another chip (control circuit chip) and used in common, not being shown in FIG. 8.
As shown in FIG. 10, a conventional neuro-computer is constructed of a plurality of neuron circuits such as shown in FIG. 8 (e.g., 300, 301, 302) and a control system 310. The plurality of neuron circuits and the control system 310 are interconnected by an input data bus 311, an address bus 312, a control signal bus 313, and an output data bus 314. Input data, address, and control signal generated by the control system are transferred to all neurons via the input data bus 311, address bus 312, and control signal bus 313, and subjected to necessary processes at the neuron circuits 300, 301, 302, . . . Thereafter, the designated neuron circuit outputs its values such as an internal state value and a propagation error to the output bus 314 via a tri-state buffer 315 or the like, and to the control system 310. The control system 310 includes a control storage unit for storing microprograms sent from a host computer, an input layer memory for storing input patterns, an desired signal memory for storing desired signals, and an output layer memory for storing the values of neurons at the output layer for all patterns.
The characteristic features of the neuro-computer shown in FIGS. 8 and 10 reside in that a learning algorithm is realized by a logic circuit (hard wired scheme) of fixed hardware. If a learning algorithm is realized by the hard wired scheme, learning can be executed at a high speed.
Next, the second conventional technique will be described. A neuro-computer has been introduced in the extra issue of Nikkei Intelligent System, Summer Edition, 1992, pp. 144-151, entitled "Parallel Machine "CNAPS System" with 256 Element Processors". This neuro-computer can assemble 256 element processors (hardware neurons) of the CNAPS architecture at a maximum, and can realize the maximum learning performance of 1.08 GCUPS. It is stated in this extra issue that the neuro-computer is applicable to learning algorithms such as BP (Back Propagation), LVQ (Learning Vector Quantization), SOM (Self Organization Map), SOM2D (Self Organization Map/2-Dimension), and FSCL (Frequency Sensitive Competitive Learning), and to applications other than neuro-applications.
According to the first conventional technique, a learning algorithm is realized by the hard wired scheme. Therefore, the learning (training) algorithm can be executed at a high speed on one hand, and on the other hand, it cannot deal with different learning algorithms (not realized by hardware). There arises a problem that the application field is limited (unable to apply to wide application fields), when taking into consideration of that an optimum learning algorithm differs depending on each application field.
There is another problem of a very expensive cost because hardware (neuro-computer) is required to be reconfigured for each learning algorithm in order to apply to wide application fields.
In order to solve the problems associated with the hard wired scheme, it is conceivable to use a microprogram scheme. Namely, a learning algorithm is microprogrammed to deal with a plurality of learning algorithms by rewriting each microprogram (without adding/modifying hardware).
However, use of the microprogram scheme poses the following problems.
(1) It takes a time to decode a microinstruction, greatly lowering an instruction execution time as compared to the hard wired scheme. PA1 (2) It is necessary to transfer data (microprogram) from a host to a neuro-computer in order to rewrite a microprogram. If rewriting a microprogram occurs frequently, the transfer time of data (microprogram) from the host to the neuro-computer becomes unnegligible, elongating the total execution time (=data transfer time+instruction execution time).
The second conventional technique has a limit in the number of neurons and in the learning performance. For a sign check which is one of applications, a desired recognition performance is impossible unless about 1000 neurons are used for the input layer, hidden layer, and output layer. In such a case, it is apparent that the number of neurons becomes insufficient if the second conventional technique is applied. Furthermore, although the second conventional technique describes that it can deal with learning algorithms other than a BP algorithms, examples of particular structures are not disclosed.